Video Coding with Adaptive Vector Quantization and Rate Distortion Optimization

The object of this dissertation is to investigate rate-distortion optimization and to evaluate the prospects of adaptive vector quantization for digital video compression. Rate-distortion optimization aims to improve compression performance using discrete optimization algorithms. We first describe and classify algorithms that have been developed in the literature to date. One algorithms is extended in order to make it generally applicable; the correctness of this new procedure is proven. Moreover, we compare the complexity of the aforesaid algorithms, first implementation-independent and then by run-time experiments. Finally, we propose a technique to speed up one of the aforementioned algorithms. Adaptive vector quantization enables adaption to sources with unknown or non-stationary statistics. This feature is important for digital video data since the statistics of two subsequent frames is usually similar, but in the long run the general statistics of frames may change even if scene changes are neglected. We examine combinations of adaptive vector quantization with various stateof-the-art video compression techniques. First we present an adaptive vector quantization based codec that is able to encode and decode in real-time using current PC technology. This codec is rate-distortion optimized and adaptive vector quantization is applied in the wavelet transform domain. The organization of the wavelet coefficients is then made more efficient using adaptive partition techniques. Moreover, the main adaptability mechanism of adaptive vector quantization, the so-called codebook update, is studied. Finally, a combination of adaptive vector quantization and motion compensation is taken into consideration. We show that for very low bitrates adaptive vector quantization performs on prediction residual frames better or at least as well as discrete cosine transform coding.